Biometric technologies have become an integral part of many secure access systems. Biometric-based authentication systems are being deployed in both low-risk secure systems, such as laptops and cell phones, and to relatively high-risk secure systems such as military bases and airports. The use of biometric technologies has a number of advantages over password or smartcard-based technologies, such as user convenience, high security, and less fraud. However, like many other authentication technologies, biometric-based systems are also vulnerable to security breaches. The cost of replacing a biometric token or template is higher to that of a password or a smart card, with severe security and privacy implications. The templates can be reused over digital networks or can be used to reproduce synthetic biometric templates such as fake fingers or model faces. In the case of face templates, there is an additional risk that the identity of a person using a biometric access system in a highly secure facility can be revealed. Several sources of security breaches in biometric-based authentication systems have been found. Some countermeasures have also been proposed to nullify such threats and the standardized biometric application programming interface (BioAPI) is continuously updated with countermeasure guidelines such as encrypting templates, avoiding storage and transmission of original templates, and performing quantization of match scores.
In general, most biometric authentication systems have four major modules: a biometric template acquisition sensor, a matching module to compare a new template to an enrolled template, a decision module using predefined thresholds for particular operational points, and a database for enrolled templates (template gallery). In many applications, it is not possible to integrate all these modules into one unit. In such scenarios, the information is passed from one unit to the other through digital channels and/or stored in digital media for offline processing. Each module possesses different levels of security threats, and different countermeasures are necessary to nullify such threats. For instance, ‘aliveness’ detection at the sensor unit will detect any attempts to hack the system with synthetic templates. Similarly, a secure database or a secure digital channel will prevent any unauthorized access to templates over a network. In applications, where the matching module and decision module are not integrated together, the ‘match scores’ must be stored in a digital media or transmitted through a digital channel to a decision module. Security breaches resulting from attacks on match scores can occur in distributed network cluster biometric systems with a central decision unit. Such networks are common in wide-area monitoring contexts.
The dominant approach for a match score-based attack on a biometric system is based on hill climbing. C. Soutar was the first to propose an iterative template adaptation method, popularly known as the hill climbing attack, to break into a biometric system based on match scores. Biometric System Security, Secure, vol. 5, p. 46-49 (2002). As shown in FIG. 1, hill-climbing approach 1 attacks the account of a subject, referred to as the targeted subject, by starting from arbitrary face template 2 and iteratively refining it in operation 3. Face recognition system (FRS) 15 outputs match score 4, which is the distance between arbitrary face template 2 and target subject 41. At every iteration, if the modified template results in a better score than the previous match score in operation 7, then the modified template is retained in operation 9, or else, it is discarded and the prior template is modified again in operation 8. The process is iterated until the template is accepted as the targeted subject in operation 6. With this method, a break-in may be achieved using a final template that does not look like any real face, as long as it deceives the system. In other words, it is not a face reconstruction method but just a break-in strategy.
One countermeasure for the first generation of hill climbing approaches is to quantize the match scores. In this approach, the FRS outputs match scores, but does not alter the match scores with small changes in input images. With appropriate quantization, it is not possible to get the incremental feedback needed by these approaches. Therefore, A. Adler developed a modified hill climbing attack for a face recognition system with quantized match scores using an additional independent set of eigenfaces. Images Can Be Regenerated from Quantized Biometric Match Score Data, Proc. Canadian Conf. Electrical and Computer Eng., p. 469-472 (May 2004). In Adler's modification, after initializing the process with an arbitrary face template, at every iteration, the previously updated template is multiplied with randomly selected eigenfaces having different weights. This generates templates farther away from the previous template. The face template that results in a better match score is retained as the updated image for the next iteration. The process terminates when there is no further improvement in match scores. Experimental results on a commercial face recognition algorithm show that after nearly 4,000 attempts, a high match score is achieved with 99% confidence. Later, Adler extended this idea to work with encrypted face templates. Vulnerabilities in Biometric Encryption System, Proc. Int'l Conf. Audio and Video-Based Biometric Person Authentication, p. 1100-1109 (July 2005).
Security breaches are possible not only in face biometrics but in other biometric applications also. U. Uludag and A. Jain extended the hill climbing approach to break into minutiae-based fingerprint recognition system. Attacks on Biometric Systems: A Case Study in Fingerprints, Proc. SPIE-EI 2004, Security, Steganography and Watermarking of Multimedia Contents, p. 622-633 (January 2004).
Although hill climbing-based attacks can successfully break a particular targeted account, effective countermeasures for such attacks can also be created. One property of hill climbing-based attacks is that they require a large number of attempts before success. Therefore, one possible countermeasure for such attacks is to restrict the number of consecutive, unsuccessful attempts. However, this still leaves the system vulnerable to a spyware-based attack that interlaces its false attempts with attempts by genuine users (successful attempts) and collects information to iterate over a period of time. However, in most hill climbing-based attacks, the templates at the ith attempt (iteration) are generated from the (i−1)th attempts (iterations) and are similar to each other. Therefore, if all unsuccessful attempts for a particular targeted account within a fixed time interval are monitored, a pattern of similar faces with decreasing dissimilarity scores will be found. Therefore, a continuous observation of unsuccessful match scores will help to detect hill climbing-based spyware attacks.
Multidimensional Scaling (MDS) has been used to derive models for standard classifiers such as nearest neighborhood, linear discriminant analysis, and linear programming problem from the dissimilarity scores between objects. E. Pekalska, P. Paclik, and R. P. W. Duin, “A generalized kernel approach to dissimilarity based classification,” Journal of Machine Learning Research, vol. 2, pp. 175-211 (2001). A similar framework has also been suggested, where pair-wise distance information is embedded in the Euclidean space, and an equivalence is drawn between several clustering approaches with similar distance-based learning approaches. V. Roth, J. Laub, M. Kawanabe, and J. M. Buhmann, “Optimal cluster preserving embedding of non-metric proximity data,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 25, no. 12, pp. 1540-1551 (2003). There are also studies that statistically model similarity scores so as to predict the performance of the algorithm on large data sets based on results on small data sets [20]-[23]. P. Wang, Q. Ji, and J. L. Wayman, “Modeling and predicting face recognition system performance based on analysis of similarity scores,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 29, no. 4, pp. 665-670 (2007); S. Mitra, M. Savvides, and A. Brockwell, “Statistical performance evaluation of biometric authentication systems using random effects models,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 29, no. 4, pp. 517-530 (2007); R. Wang and B. Bhanu, “Learning models for predicting recognition performance,” IEEE International Conference on Computer Vision (ICCV), pp. 1613-1618 (2005); and G. H. Givens, J. R. Beveridge, B. A. Draper, and P J Phillips, “Repeated measures glmm estimation of subject-related and false positive threshold effects on human face verification performance,” IEEE Conference on Computer Vision and Pattern Recognition-Workshops p. 40 (2005). For instance, Grother and Phillips proposed a joint density function to independently predict match scores and non-match scores from a set of match scores. P. Grother and P. J. Phillips, “Models of large population recognition performance,” In Proc. of IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 68-75 (2004). Apart from face recognition, methods have been proposed to model and predict performances for other biometric modalities and objects recognition. M. Boshra and B. Bhanu, “Predicting performance of object recognition,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 22, no. 9, pp. 956-969 (2000) and D. J. Litman, J. B. Hirschberg, and M. Swerts, “Predicting automatic speech recognition performance using prosodic cues,” First conference on North American chapter of the Association for Computational Linguistics, pp. 218-225 (2000).
Recently, a method of modeling a face recognition algorithm using an affine transform was developed. P. Mohanty, S. Sarkar, and R. Kasturi, Designing Affine Transformations Based Face Recognition Algorithms, Proc. IEEE Workshop Face Recognition Challenge (June 2005). Starting from distances computed by any face recognition algorithm, such as the Face Recognition Grand Challenge (FRGC) baseline algorithm, the modeling process calculates the best affine transform that approximates it. The modeling process is a closed-form solution based on classical Multidimensional Scaling (MDS).
Attempts to find vulnerabilities have focused on modifications of the hill-climbing technique; however, these techniques have become identifiable by recognition systems as attacks because of their iterative nature. Discovery of vulnerabilities in recognition systems, therefore, needed to be expanded beyond variations of the hill-climbing technique, in order for countermeasures to be designed to further prevent security breaches resulting from a recognition system's vulnerabilities. Although a process of modeling a face recognition algorithm was available, the process needed modification and improvement to better model the FRS and a method was needed to utilize the improved modeling process to identify vulnerabilities in face recognition systems.
To meet this need, a non-iterative method of reconstructing unknown image templates of biometric systems using match scores was developed. P. Mohanty, S. Sarkar, and R. Kasturi, From Scores to Face Templates: A Model-Based Approach, IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 29, No. 12 (December 2007). The non-iterative method is the subject of U.S. patent application Ser. No. 12/187,028, which is herein incorporated by reference. This method provides a way of reconstructing biometric image templates using the match scores of distinct images from a face recognition system (FRS). When the reconstruction method ‘breaks in’ to a FRS, it creates no obvious patterns in the match scores, preventing detection by the FRS. Known countermeasures are not effective in preventing security breaches caused by the method, and new countermeasures will be difficult to design because of the absence of an obvious pattern in the match scores.
FIG. 2 is a schematic of the search process illustrating the differences between hill climbing attack 1 and reconstruction method 10. Reconstruction method 10 requires the distances, or match scores, between the image of targeted subject 9 and a set of distinct images from break-in set 17A-17D. Whereas, a hill climbing-based attack computes scores for faces along a trajectory of incremental scores from arbitrary template 2 to that of targeted subject 9. The statistically decreasing dissimilarity scores generated by a hill climbing-based approach can be used to detect such attacks, but such detection strategies cannot be applied to reconstruction method 10. Hill climbing attack 1 is a ‘break-in’ strategy to a FRS, whereas reconstruction method 10 is a template reconstruction method. In addition, attacks on an FRS are more feasible in real-time applications by reconstruction method 10, because the number of attempts is predefined by the number of images in the break-in set, instead of potentially unlimited, as in hill climbing attack 1.
A simplified diagram of reconstruction method 10 is shown in FIG. 3. A set of face images different from the gallery and probe sets is used as ‘break-in’ set 17. Modeling process 20 creates a model of FRS's 15 face recognition algorithm. FRS 15 is treated as a complete black box and no reverse engineering is performed on it. Instead, an assumption is made as to the type of face recognition algorithm used by FRS 15. It may be possible to identify the recognition algorithm given score matrices of known algorithms, but this is not necessary for the success of reconstruction method 10.
Modeling process 20 is an offline procedure and needs to be constructed only once for a given recognition algorithm. Once the model is built, templates are presented from break-in set 17 to FRS 15, which calculates match scores 46 to unknown target template 41. Unknown target template 41 represents an enrolled template in the FRS, which when reconstructed results in a successful ‘break-in’. Embedding process 40 then uses match scores 46 to embed unknown target template 41 in the model. Finally, template construction process 60 manipulates the model to construct unknown target template 41.
Although this process of modeling a face recognition algorithm was available, the process needed modification and improvement to more efficiently model the FRS.